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ABSTRACT 

The ubiquity of online programs in higher education requires continued focus on designing 
instructional environments that improve students' learning. We examine students' perceptions 
of sense of community and learning, as well as academic achievement, using grades obtained 
from a final project and participation in asynchronous discussion forums. Findings indicate a 
significant correlation between perceived learning and the sense of community connectedness 
subscale. Although sense of community is closely associated with interactions, the results did 
not show a significant relationship between the sense of community and the discussions 
achievement variable. Implications and challenges in implementing activities to foster sense 
of community in an online learning environment are discussed. 

Keywords: Sense of community, perceived learning, achievement, online learning, 
asynchronous discussions. 

INTRODUCTION 

In the United States, 7.1 million higher education students took at least one online course 
during 2012. This represents a 6.1 % growth rate from the previous year (Allen & Seaman, 
2014). Furthermore, in 2012, 62.4% of higher education institutions offered online courses 
and full programs as compared to 34.5% in 2002 (Allen & Seaman, 2013). The growth of online 
programs in higher education requires continued attention to the design of instructional 
environments to improve learning. Among different conceptions of learning, constructivist 
theories emphasize the creation of knowledge by the learners while they attempt to make 
sense of their experiences through interactions with the community and the environment 
(Driscoll, 2005; Harasim, 2012). Among the many factors that may impact the success of online 
environments, the development of online communities has become an important field of 
interest, especially in higher education (Bond & Lockee, 2014; Shea, 2006; Snyder, 2009). 

Based on collaborative constructivism. Garrison's (2011) Community of Inquiry theoretical 
framework (Col) identified social presence, cognitive presence, and teaching presence as key 
elements to inform instructional design and create meaningful e-learning experiences 
(Richardson, et al., 2012). One important factor positively associated with Col is sense of 
community (SOC) (Garrison, 2007). In the community psychology field, SOC was defined as 
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the feeling of belonging to a group, the sense that group members mattered to each other and 
would meet each other's needs through their shared commitment to the group (McMillan and 
Chavis, 1986)."the feeling that members have of belonging, a feeling that members matter to 
one another and to the group, and a shared faith that members' needs will be met through 
their commitment to be together" (McMillan and Chavis, 1986, p. 9). Extending the concept to 
an educational setting, Rovai (2002b) defined classroom community as 

a feeling that members have of belonging, a feeling that members matter to 
one another and to the group, that they have duties and obligations to each 
other and to the school, and that they possess shared expectations that 
members' educational needs will be met through their commitment to shared 
learning goals. One can, therefore, constitutiveiy define classroom 
community as consisting of two components: feelings of connectedness 
among community members and commonality of learning expectations and 
goals, (p. 322) 

Many scholars discuss the relevance of building sense of community in online educational 
environments to improving students' overall satisfaction with the learning experience 
(Garrison, 2007, 2011; Moore, 2014; Palloff & Pratt, 2007a, 2007b; Shackelford 8i Maxwell, 
2012; Stepich & Ertmer, 2003). There is evidence that SOC is positively related to other 
variables such as perceived learning (Liu, Magjuka, Bonk & Lee, 2007; Rovai, 2002b; Shea, 
2006; Shea, Li, & Pickett, 2006; Top, 2012), satisfaction (Drouin, 2008; Tsai et al., 2008; Ouzts, 
2006), engagement (Young & Bruce, 2011), and achievement (Harvey, Moller, Huett, Godshalk 
& Downs, 2007; Wighting, Nisvet & Spaulding, 2009). For example, Wighting, Nisvet and 
Spaulding (2009) found a relationship between the community learning subscale and 
academic achievement, and between SOC and academic achievement. Specifically in online 
courses in instructional design, Ertmer and Stepich (2005) found a significant relationship 
between perceived learning and the learning community subscale, but not between SOC and 
learning. Thus, as Wighting, Nisvet and Spaulding (2009) concluded, "learning has important 
social and cognitive dimensions and occurs most effectively when the school provides a 
positive social environment with a strong sense of community" (p. 64). 

PROMOTING SENSE OF COMMUNITY 

There have been several attempts to identify general recommendations to promote SOC. 
Palloff and Pratt (2007a) suggested active interaction, collaborative learning, socially 
constructed meaning, resource sharing, and expressions of support and encouragement. 
Bielaczyc and Collins (1999) identified three similar strategies that activities should have in 
order to promote learning communities in the classroom: providing individual development 
and collaborative construction of knowledge, sharing knowledge and skills among members 
of the community, and making learning processes visible. Haythornthwaite, Kazmer, Robins, 
and Shoemaker (2000) also suggested three instructional strategies to promote SOC in 
distance education: promoting initial bonding, monitoring and supporting interaction and 
participation, and providing multiple ways of communication. 

SOC may rise when students have opportunities to interact with their classmates and 
instructor (Cameron et al., 2009; Dawson, 2006; Drouin, 2008; Ouzts, 2006; Shen et al., 2008; 
Swan, 2002). Focusing on the types of learner-learner interactions, Shackelford and Maxwell 
(2012) found that the following activities, in order of relevance, have an impact on developing 
students' SOC in online learning: introductions, collaborative group projects, contributing 
personal experiences, entire class online discussions, and exchanging resources. Applying 
guidelines proposed by Palloff and Pratt (1999) to their own online courses on instructional 
design, Stepich and Ertmer (2003) proposed the following: Instructors should promote 
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community-building from the very beginning of the course, monitor and support participation 
throughout the semester, focus instructor participation on providing "weaving" comments, 
measure perceptions of "community" within the class, and create a technology "boot camp" to 
ensure that everyone can use the technology easily. In summary, Snyder (2009) stated, "As 
online learning matures, it is important for both theorists and practitioners to understand how 
to apply new and emerging educational practices and technologies that foster a sense of 
community and optimize the online learning environment" (p. 48). 

The purpose of this study is to extend the results of previous research analyzing relationships 
between sense of community, perceived learning, and academic achievement in an online 
graduate course in instructional design. Using different activities recommended in the 
literature to promote SOC, this study explores students' increase in instructional design skills 
during an eight-week online course, examining how SOC is related to two measures of 
achievement and students' perceptions of community. This study addresses the following 
research questions: 

> Is there a relationship among sense of community, perceived learning and 
achievement in an online graduate course? 

> What do students value the most in an online graduate course that promotes 
learning and sense of community? 

METHODS AND PROCEDURES 

Participants were graduate students enrolled in an online course in instructional design during 
the 2014 summer session. The course is a required graduate course for obtaining a master's 
degree in educational technology. The first author, the instructor of the course, sent invitations 
to participate in the research study to the 21 students enrolled in this eight-week section. 
Fifteen students (10 female; 5 male) agreed to participate. Based on their self-introductions 
at the beginning of the course, students had a broad range of backgrounds, knowledge, and 
teaching experiences. 6 of them were secondary teachers, 3 students worked as elementary 
teachers, 5 were educational technology coordinators, and 1 student was a college professor. 
Assignment submissions, discussion forums, and grading took place in Moodle, the program's 
learning management system. 

Different collaborative activities were designed to promote SOC and foster learning among 
students. First, a VoiceThread presentation was created at the beginning of the course to allow 
students to introduce themselves using video. VoiceThread is a web-based tool that provides 
a multi-modal asynchronous communication among users including text, voice, and audio. 
Documentation of the tool's implementation in a graduate course has been provided elsewhere 
(Ching & Hsu, 2013). Students were also required to post comments, interacting with at least 
three classmates who shared something in common with them. As discussed by Rovai (2007), 
this initial forum was developed with the intention of promoting community building and social 
presence among students. 

Second, four asynchronous text-based discussions were scheduled to support interaction, 
participation and collaborative construction of meaning. There were two types of activities 
during discussions. At the beginning of the course, four groups were created to help the 
instructor lead the asynchronous discussion forums. These leading groups were in charge of 
preparing a VoiceThread presentation to summarize and discuss assigned content. 
Presentations were posted in Moodle on Mondays for the whole class to view. Then, during the 
discussion week, two questions were posted in the forum. Those students who were not in the 
leading groups were required to answer both questions by Thursday and write follow-up posts 
by Sunday in response to their classmates' questions or comments. 
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Third, an instructional design (ID) project was created to help students apply ID concepts to 
real-world situations. This project was divided into two sections due at different points in the 
semester. The first section required students to conduct a needs analysis. In their second 
report, students revised their analysis, developed their projects, and conducted an evaluation. 
To provide another venue for learning and supporting each other, a peer review activity was 
created. For this activity, students were assigned one classmate's first report and asked to 
provide feedback using a form (See appendix A). A course evaluation survey asking about the 
most valuable aspect(s) of the overall learning experience served as a secondary data source, 
and results were used to support findings. 

Data Collection and Analysis 

Quantitative data were collected using two surveys (perceived SOC and perceived learning) 
and by analyzing scores for the final ID projects and discussion participation. This was 
supplemented with qualitative data collected from responses to one item in the course 
evaluation survey. Although there are several instruments to measure SOC in education, most 
of them were developed for K-12 environments (Rovai, Wighting, & Lucking, 2004). Thus, for 
this study SOC was measured using the Classroom Community Scale survey, which focuses on 
postsecondary students. This survey has 20 Likert-scale questions composed of two subscales 
of connectedness and learning. Rovai (2002b) explains. 

Connectedness represents the feelings of students regarding their cohesion, spirit, trust, and 
interdependence. Learning represents the feelings of students regarding the quality of their 
construction of understanding and the degree to which they share values and beliefs 
concerning the extent to which their learning goals and expectations are being satisfied, (p. 
325) 

The instrument has questions such as: "I feel that students in this course care about each 
other, " "I feei reluctant to speak openly," and "I feel uncertain about others in this course." 
Face validity and construct validity have been established (Rovai, 2002a; Rovai & Baker, 2005). 

To measure perceived learning, students completed a self-assessment questionnaire at the 
beginning and end of the online course in which they rated themselves on 19 instructional 
design competencies. These competences were composed by the International Board of 
Standards for Training, Performance and Instruction (IBSTPI, 2014), providing content 
validity to this instrument (Leedy & Ormrod, 2013). Students were asked to read each 
competency carefully and choose the option that indicated how they perceived their level of 
competency from 1 (Weak), 2 (Somewhat weak), 3 (Neither weak or strong), 4 (Somewhat 
strong), and 5 (Strong). A gain score was calculated to determine changes in the perceived 
learning of students' ID competency. This questionnaire measured items such as the ability to 
"conduct a needs assessment in order to recommend appropriate design solutions and 
strategies," "use an instructional design and development process appropriate for a given 
project," and "evaluate instructional and non-instructional interventions." It is important to 
emphasize that most of these instructional designer competences guide the requirements of 
the instructional design project required of the students. 

Finally, achievement was measured using two different grades. The first was obtained from 
students' work on midterm and final project reports. This instructional design project assessed 
students' capability in applying instructional design concepts to an instructional design 
problem, creating different analyses, developing materials, and working with a subject matter 
expert to evaluate their projects. The second score was taken from an average of the three 
grades that students received through their participation in three asynchronous discussions. 
Discussions assessed students' active participation in the course, measuring students' 
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understanding of the reading materials related to instructional design. Final grades were 
assigned based on three rubrics created for these activities (See Appendices B, C and D). 

RESULTS AND DISCUSSION 

Because of the small sample size, a Shapiro-Wilk test was used to assess the normality of the 
data (Sen & Srivastava, 1990). Results show the SOC (W=.99, p=.99), SOC_Learning (W=.95, 
p=. 59), and perceived learning (W=.95, p=.53) are approximately normally distributed; but 
SOC connectedness (W=.82, p=.006), ID project (W=0.72, p=.001), and discussions (W=.67, 
p=.001) are not. 

Perceived Learning 

The Cronbach's alpha coefficients for the pretest and posttest were 0.92 and 0.95 respectively, 
indicating an acceptable level of reliability (Nunnally & Bernstein, 1994). Results from the self- 
assessment questionnaire showed a significant increase (f=4.36, p=.001) in students' 
perceptions of their instructional design skills from pre- to post-test scores. On average, 
students' ratings changed .86 points from 2.94 (neither weak nor strong) to 3.80 (somewhat 
strong). This result shows that the different activities designed in this course helped students 
to perceive that they gained knowledge based on instructional design competencies composed 
by the International Board of Standards for Training, Performance and Instruction (IBSTPI, 
2014). 

Sense of Community 

The Cronbach's alpha coefficient from the data was .84, indicating an acceptable level of 
reliability (Nunnally & Bernstein, 1994). Scores of students (Table 1) on the Community Scale 
averaged 71/100 points suggesting that students "agreed" that they felt SOC in the classroom. 
Additionally, students also agreed that they felt connected to each other (x=37.00), and the 
class as a community enabled them to reach their learning goals (*=34.27). 


Table: 1 

Descriptive statistic of the SOC survey 


Items 

Mean* 

Standard Deviation 

Classroom community 

71.27 

8.15 

Connectedness subscale 

37.00 

5.50 

Learning subscale 

34.27 

4.40 


* Total possible classroom community scores range from 0 to 100, with higher scores reflecting 
a stronger sense of community. Connectedness and learning subscale scores can each range 
from 0 to 50. 
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Achievement 

Distribution of the grades for discussions and the final project are shown in Table 2. Because 
both variables are measured in ratio scale but are not normally distributed, a Spearman 
correlation coefficient was used to address the relationship between the two achievement 
scores (Vaughan, 2001). The Spearman correlation showed a significant relationship between 
discussions grades and instructional design project grades (/v=. 52; pc.05). These results 
suggest that although they are two different types of assessments, they both are valid 
strategies to measure achievement in novice instructional design students. 


Table: 2 

Grades frequencies 

Scores 

Discussion grades 

ID Project grades 

0-79 

1 (6.7%) 

2 (13.3%) 

80-89 

1 (6.7%) 

3 (20%) 

90-100 

13 (86.6) 

10 (66.7%) 


Relationships among variables 

Because the data from the SOC connectedness scale, ID project, and discussions variables did 
not show a normal distribution in the Shapiro-Wilk's test, a Spearman rank-order correlation 
coefficient was used to address the relationship among the variables (Table 3). Results showed 
that for an alpha level of .05, the correlation between perceived learning and SOC in the 
connectedness subscale for the 15 students was statistically significant (/v=. 62; pc.05). Thus, 
increases in students' perceived learning were correlated with increases in students' feelings 
of connectedness. This result, a difference from what was found by Ertmer and Stepich (2005), 
suggests that students with higher gains in perceived learning found it more relevant to feel 
connected with the group than to learn from the group. Additionally, a correlation between 
perceived gain and the instructional design project measure of achievement was also 
statistically significant (/s=.53; pc.05). This result was expected, since the requirements of 
the instructional design project were guided by the instructional designer competences. 
Finally, although sense of community is closely associated with interactions (Dawson, 2006; 
Shen et al., 2008), the results did not show a significant relationship between the sense of 
community and the discussions achievement variable. 

Table: 3 

Correlations among variables 



Perceived learning 

Discussions 

ID Project 

SOC 

.45 

.18 

.17 

SOC-Connectedness 

.62* 

.32 

.38 

SOC-Learning 

.24 

-.30 

.10 


* Significant at the .05level 
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In the course evaluation survey, students were asked the following question: Which aspects 
of this course were most valuable to your overall learning experience? Using deductive coding 
to focus on the "key variables that the researcher brings to the study" (Miles, Huberman & 
Saldana, 2014, p. 81), students' responses were analyzed based on perceived learning, 
achievement (ID project and discussions) and sense of community to observe the relationships 
between them (Table 4). Additionally, the criteria of selection for these categories were 
extended to "latent content" (Berg, 2008) where participants' responses are interpreted. In 
this way, the frequency of each category observing both the manifest and the latent content 
is reported. Within a message, if a participant wrote two sentences with the same meaning, it 
was counted as one. In addition, some responses could be classified in different categories, so 
each appearance was counted as one for each category. A complete student's response was 
used as the unit of coding. 


Table: 4 

Deductive coding of the course most valuable aspects 


Variables (n=17) 

Sample Responses 

Percentage of students 
mentioning this aspect 

Perceived knowledge 

I have a working knowledge of what ID is. 

6 (30%) 

Achievement- 

Discussions 

Discussion questions related to ID process. 

5 (25%) 

Achievement-ID 

Project 

The Instructional Design Project 

4 (20%) 

Sense of Community 

Dr. T was always willing to reach out to support 
me. He was very flexible and took personal 
situations into consideration. 

4 (20%) 

Others 

None 

1 (5%) 


Overall, ninety-five percent of student's responses were classified in any of the categories. 
After analyzing the responses, the most frequently mentioned aspect (30%) is perceived 
learning. Students highlighted the learning they obtained as the most important aspect of the 
course. Looking at the relationship among the variables, one student commented that "The 
discussion groups were very valuable in processing the course material." Similarly, another 
student stated that the most valuable experience in the course was 'The leading group 
Voicethread group discussion. This allowed students to work together, help each other, and 
feel a sense of accomplishment." As such, participants especially appreciated interacting with 
peers to work on the different learning activities. Thus, results from the SOC survey and 
students' comments support the use of collaborative activities such as leading groups and 
discussions to help students to build SOC in the group. 

CONCLUSIONS AND IMPLICATIONS 

The use of different activities to promote SOC (Haythornthwaite, et al., 2000; Palloff and Pratt, 
2007a; Rovai, 2007) is well documented. In this study, collaborative activities were designed 
to promote SOC among students. First, an introductory activity was created where students 
introduced themselves and interacted with their classmates. Second, leading groups were 
created to develop a presentation summarizing specific content from the course and to help 
lead a one-week discussion. Third, four whole-class discussion forums were developed to 
discuss relevant ID concepts. Finally, students participated in a peer-review activity providing 
feedback on one classmate's instructional design project. 
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The results of this study suggest that different activities designed for this online instructional 
design graduate course are associated with students' perceived learning and a positive sense 
of community. Students' comments from the course evaluation about the aspects of this course 
which were most valuable to their overall learning experience helped us to confirm that the 
activities were beneficial for them. Besides significant correlations between perceived learning 
and sense of connectedness, and between perceived learning and grades from the final project, 
results did not show significant correlations between the two achievement measures and SOC. 
In a similar study conducted on students enrolled in instructional design courses, Ertmer and 
Stepich (2005) used bulletin board postings and students' case analysis as activities to 
measure students' achievement. Their results also suggested a lack of any significant 
relationship between these activities and SOC. We echo their conclusion that additional 
research should be conducted using other type of activities to measure learning and/or 
achievement. 

Because of the rather limited sample size and specific learning contexts and tasks, findings in 
this study should be interpreted with caution. Since individual characteristics appear to relate 
to differences in perceived SOC (Drouin & Vartanian, 2010), future research should replicate 
these type of questions in online courses with different learners, such as undergraduates from 
different fields, in order to corroborate or contradict findings related to SOC. Future research 
could also incorporate interviews with some of the participants to provide insights into how 
they value each one of the activities designed in this course. Configuration of the different 
activities could be modified as well. For example, discussion forums were one-week long and 
students received full credit when they wrote at least four posts. Longer periods of time and 
an undetermined amount of postings required may increase the quantity and the quality of 
postings (Gilbert & Dabbagh, 2005). Additionally, although VoiceThread is an excellent tool to 
motivate interactions and social presence, this Web 2.0 tool needs to be improved to allow 
extended threaded conversations (Borup, West & Graham, 2012). 

The results have implications for instructors who teach online, especially for those looking for 
activities regarding how to promote SOC. As part of the goal of increasing teaching presence 
(Shea, Li, Pickett, 2006; Shea, Li, Swan & Pickett, 2005), instructors have the responsibility to 
design and organize e-learning environments to facilitate collaborative communities of inquiry 
(Richardson, et al., 2012). Different types of activities, like the ones proposed in this study, 
provide examples that may help instructors to elicit sense of community and learning among 
students in their own online courses. Furthermore, instructors in the instructional design field 
may also be interested in how activities such as instructional design projects, peer-review 
activities, and asynchronous discussions may help students to improve achievement and their 
perceived learning. 
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Appendix A: Peer Feedback Form 
Report #1 Formative Assessment: 

Please, read carefully your classmate's ID project and provide detailed and thoughtful 
feedback on each one of the sections to help your classmate to improve his/her ID project. 
Make sure your comments describe the strengths of the project and details where the project 
can be improved based on our readings (Larson & Lockee's textbook) and the information on 
the ID Project Outline document. 

Part 1. Topic 

> Goal statement: 

The goal is expressed in clear, unambiguous terms; it is succinct and not too broad. 
Comments: 

> Audience description: 

The audience (learners) is clearly described. Comments: 

Part 2. Analysis Report 

> Gather data on needs, resources, and constraints: 

> Needs assessment survey and other documentation: Includes both a description of 
the survey questionnaire and the actual questions. Describe here any other 
documentation you used to verify the needs. Comments: 

> Needs assessment data: Describes the learners' needs with a rich level of detail 
based on results of the analysis survey. Comments: 

> Learner needs and characteristics 

Describes the characteristics of the learners with a rich level of detail. Comments: 

> Learning context description 

Describes the features of the learning context with a rich level of detail. Comments: 

> Context analysis 

Describes the features of the transfer context with a rich level of detail. Comments: 

> Content analysis (Flow diagram) 

Highly detailed, organized chart that shows very finite description of breakdown of 
tasks related to the learning goal and objectives. Comments: 

Overall Review 

Make any additional comments about the first report and provide specific suggestions and 
recommendations to improve the quality and clarity of this instructional design project. 
Comments: 
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Appendix B: Midterm Report Rubric 


Items 

Exceed expectation 

Meet expectation 

Below expectation 

Goal statement 
(10%) 

The goal is expressed 
in very clear and 
unambiguous terms; it 
is succinct and 
describes a realistic 
and focused project. 

The goal is expressed 
in clear and 
unambiguous terms; it 
is succinct and 
describes a realistic 
and focused project. 

The goal is unclear 
and the project is too 
broad in scope. 

Audience 

description 

(10%) 

The audience 
(learners) is clearly 
described; it contains 
two or three sentences. 

The audience 
(learners) is clearly 
described; it contains 
two or three 
sentences. 

The audience 
(learners) is undearly 
described. 

Needs 
assessment 
survey (15%) 

Includes both a very 
clear description of the 
survey questionnaire 
and the actual 
questions. Survey has 
at least 15-20 
questions, and the 
questions are 
thoughtfully designed 
to meet the needs of 
the project. Includes a 
description of how the 
survey was conducted 
and how many learners 
received/filled out 
surveys. 

Includes both a 
description of the 
survey questionnaire 
and the actual 15-20 
questions. Survey has 
at least 15-20 
questions, and the 
questions meet the 
needs of the project. 
Includes a description 
of how the survey was 
conducted and how 
many learners 
received/filled out 
surveys. 

Description of the 
survey is unclear, 
survey has less than 

15 question, and/or 
actual survey 
questions were not 
provided. If provided, 
the survey questions 
do not seem relevant 
to the project. No 
information about 
how the survey was 
conducted. 

Needs 
assessment 
data (15%) 

Describes the learners' 
needs in a narrative 
format with a rich level 
of detail based on 
results of the analysis 
survey. It includes at 
least one relevant 
graph. 

Describes the learners' 
needs in a narrative 
format with a good 
level of detail based 
on results of the 
analysis survey. It 
includes at least one 
relevant graph. 

Cursory analysis of 
the learners' needs 
and it is not 
presented in a 
narrative format. No 
graphs were included. 

Learner needs 
and 

characteristics 

(15%) 

Describes learners' 
characteristics with a 
rich level of detail. 
Description includes 
learner demographics 
and prior skills. It 
includes at least one 
relevant graph. 

Describes learners' 
characteristics with a 
good level of detail. 
Description includes 
learner demographics 
and prior skills. It 
includes at least one 
relevant graph. 

Cursory description of 
learners' 

characteristics. No 
graphs were included. 

Learning 
context (15%) 

Describes the features 
of the learning context 
with a rich level of 
detail. Examines the 
context from a critical 
perspective. 

Describes the features 
of the learning context 
with a good level of 
detail. Examines the 
context from a critical 
perspective. 

Cursory description of 
the learning context. 
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Performance 
context (10%) 

Describes the features 
of the transfer context 
with a rich level of 
detail, addressing how 
students will use what 
they have learned 
outside the classroom 

Describes the features 
of the transfer context 
with a good level of 
detail, addressing how 
students will use what 
they have learned 
outside the classroom 

Cursory description of 
the performance 
context. 

Content 

analysis-Flow 

diagram 

(10%) 

Highly detailed and 
organized chart that 
shows a very finite 
description of 
breakdown of tasks 
related to the learning 
goal and objectives 

Detailed and 
organized chart that 
shows a finite 
description of 
breakdown of tasks 
related to the learning 
goal and objectives 

Unclear chart with 
missing detailed steps 
that makes it difficult 
to understand. 
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Appendix C: Final Report Rubric 


Items 

Exceed expectation 

Meet expectation 

Below expectation 

Updated 
report #1 
(10%) 

All suggested changes in 
the feedback have been 
very well addressed and 
all changes improve the 
overall quality of the 
instructional design 
document 

All suggested changes in 
the feedback have been 
addressed and changes 
improve the overall 
quality of the 
instructional design 
document 

Some of the 
suggested changes 
in the feedback 
have been 
addressed 

Rationale 

(10%) 

A very well written, 
detailed explanation of 
the need for the project. 

It identifies needs, 
describes overall 
scaffolding strategy and 
pedagogical approach. 
Each one of these 
descriptions explains 
why it can be classified 
as such, and contains 
good detail and 
adherence to text 
without being wordy. 

A well-written 
explanation of the need 
for the project. It 
identifies needs, 
describes overall 
scaffolding strategy and 
pedagogical approach. 
Each one of these 
descriptions explains 
why it can be classified 
as such. 

Limited information 

about the need the 
project meets. It 
does not describing 
clearly overall 
scaffolding 
strategy, 
pedagogical 
approach, and why 
it can be classified 

as such. It contains 

mechanical errors. 

Learning 

objectives 

(5%) 

A comprehensive set of 
objectives and sub¬ 
objectives that are 
numbered and very 
detailed about the 

instructional outcomes. 
Objectives are well 
written in a clear, 
unambiguous manner 
including conditions, 
behavior, and criteria. 

A comprehensive set of 
objectives and sub¬ 
objectives that are 
numbered and detailed 
about the instructional 
outcomes. Objectives 
are written in a clear, 
unambiguous manner. 

List of objectives is 
very short or very 
long, lacking in 
detail, and does not 
include sub¬ 
objectives. 

Matrix of 

objectives. 

Bloom's 

taxonomy, 

and 

assessments 

(5%) 

Table is clearly filled out 
with proper 
identification of 
taxonomy level, 
scaffolding strategy, and 
assessment types. 

Table is filled out with 
proper identification of 
taxonomy level, 
scaffolding strategy, and 
assessment types. 

Table provided is 
not filled out with 
proper 

identification of 
taxonomy level, 
scaffolding 
strategy, and 
assessment types. 

ARCS table 
(5%) 

Chart clearly identifies 
the motivational 
strategies to be used for 
each of the four areas 
with a rich level of 
detail. 

Chart identifies the 
motivational strategies 
to be used for each of 
the four areas. 

Chart does not 
identify clearly the 
motivational 
strategies to be 
used for each of the 
four areas. 
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Instructor 
guide (10%) 

It is very detailed and 
makes flow of 
instruction very clear 
following the Instructor 
Guide Outline. If a 
flowchart is provided, it 
is very clear and makes 
good use of color, and 
logical flow. 

It is very detailed and 
makes flow of 
instruction clear 
following the Instructor 
Guide Outline. If a 
flowchart is provided, it 
is clear and makes good 
use of color, and logical 
flow. 

It does not contain 
enough details and 
makes flow of 
instruction unclear. 

If a flowchart is 
provided, it is not 
very clear. 

Learner 

Content: 

Learning 

materials 

(10%) 

Materials are very well 
developed (if not in final 
state, close to it), 
amount of material is 
sufficient to support 
stated goals/objectives, 
and materials are 
professional in 
appearance. Outside 
resources are cited 
correctly in a reference 
page. 

Materials are well 
developed (if not in final 
state, close to it), 
amount of material is 
sufficient to support 
stated goals/objectives, 
and materials are 
professional in 
appearance. Outside 
resources are cited 
correctly. 

Materials are not 
well developed, 
amount of material 
is not sufficient to 
support 

goals/objectives, 
and materials are 
not professional in 
appearance. 

Outside resources 
aren't cited 
correctly. 

Learner 

Content: 

Formative 

and/or 

Summative 

Assessment 

(10%) 

Formative and/or 
summative assessments 
"fit" the type of learning 
identified and 
instructional strategies 
proposed. If adapted, 
they are well 
documented. 

Formative and/or 
summative assessments 
"fit" the type of learning 
identified and 
instructional strategies 
proposed. If adapted, 
they are well 
documented. 

Formative and/or 
summative 
assessments do not 
"fit" the type of 
learning identified 
and instructional 
strategies 
proposed. Materials 
are not 
documented. 

Evaluation 
plan (10%) 

A comprehensive plan to 
evaluate the 
instructional design 
project using 

Kirkpatrick's model is 
clearly described. Each 
one of the four levels 
includes key questions 
you would want 
answered at each level, 
who would be involved, 
possible measurement 
instruments, and any 
additional information 
that helps to describe 
your evaluation plans. 

A plan to evaluate the 
instructional design 
project using 

Kirkpatrick's model is 
described. Each one of 
the four levels includes 
key questions you would 
want answered at each 
level, who would be 
involved, possible 
measurement 
instruments, and any 
additional information 
that helps to describe 
your evaluation plans. 

A plan to evaluate 
the instructional 
design project 
using Kirkpatrick's 
model is not clearly 
described. Not all 
four levels include 
key questions that 
would be answered 
at each level, who 
would be involved, 
and possible 
measurement 
instruments. 

Subject 

Matter Expert 
(SME) (5%) 

Clearly indicates who 
has served as the SME 
identifying with rich 
level of detail his/her 
expertise in the project 
content 

Clearly indicates who 
has served as the SME 
identifying his/her 
expertise in the project 
content 

SME is described 
but not named 
and/or it is not 
clear his/her 
expertise in the 
project content. 
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Evaluation 
survey/rubric 
(5%) 

A SME evaluation rubric 
or survey is included to 
properly evaluate the 
materials created. 
Questions are well- 
designed to obtain a 
thorough evaluation. If 
it is a survey, yes/no 
questions are avoided to 
obtain more detailed 
responses. 

A SME evaluation rubric 
or survey is included to 
properly evaluate the 
materials created. 
Questions are well- 
designed to obtain a 
thorough evaluation. If 
it is a survey, yes/no 
questions are avoided to 
obtain more detailed 
responses. 

A SME evaluation 
rubric or survey is 
included but it 
doesn't evaluate 
properly the 
materials created. 

If it is a survey, 
yes/no questions 
were included. 

Expert review 
report (5%) 

Clear and detailed 
presentation of SME's 
evaluation of the 
materials that 
summarizes the most 
important points. It is 
written in narrative form 
and does not exceed one 
page. 

Clear presentation of 
SME's evaluation of the 
materials that 
summarizes the most 
important points. It is 
written in narrative form 
and does not exceed one 
page. 

Report is vague 
and/or short, and 
does not include a 
clear presentation 
in a narrative form 
of the SME's 
evaluation. 

Comments on 
suggestions 
(10%) 

Carefully dissects any 
feedback/constructive 
criticism from SME and 
addresses in detail how 
the concerns would be 
addressed in future 
iterations of the 
instructional materials; 
does not exceed one 
page. 

Carefully dissects any 
feedback/constructive 
criticism from SME and 
addresses how the 
concerns would be 
addressed in future 
iterations of the 
instructional materials; 
does not exceed one 
page. 

Cursory analysis on 
how the concerns 
would be addressed 
in future iterations 
of the instructional 
materials. 
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Appendix D: Asynchronous Discussion Rubric 


Items 

Exceed expectation 
(2) 

Meet Expectation (1) 

Below Expectation (0) 

Quality of 
initial posts 
(20%) 

The initial posts are 
well-written and 
answers the 
discussion 
question(s) 
thoughtfully 
addressing all aspect 
of the question. 

The initial posts are 
well-written and 
answers the discussion 
question(s) but lacks to 
fully address all aspects 
of the questions. 

No initial posts are made 
or the initial posts contains 
superficial thought and 
preparation and does not 
address all aspects of the 
questions. 

Quality of 
follow-up 
posts 
(20%) 

Follow-up posts 
answer completely 
classmates' 
question(s) to your 
initial posts and offer 
useful suggestions 
and new perspectives 
on classmates' initial 
posts for further 
development of the 
discussion. 

Follow-up posts 
answer classmates' que 
stion(s) and some of the 
comments offer useful 
suggestions and new 
perspectives for further 
development of the 
work. 

Most of the follow-up 
comments are shallow 
contributions to the 
discussion (e.g., great job!) 
or not helpful for further 
development of the work. 

Content 

contribution 

(20%) 

Posts a factually 
correct, reflective, 
and substantive 
contribution that 
advances discussion. 

Posts information that is 
factually correct but 
lacks full development 
of concept or thought. 

Posts information off is off 
the topic or irrelevant and 
does not add substantive 
information to the 
discussion. 

References 
and support 
(20%) 

Posts contain 
references to the 
literature, readings, 
or personal 
experience to support 
students' answers. 

Posts incorporate some 
references from 
literature and personal 
experience. 

Includes no references or 
supporting experience. 

Clarity and 
mechanics 
(20%) 

Contributes to 
discussion with clear, 
concise comments 
formatted in an easy- 
to-read style free of 
grammatical or 
spelling errors. 

Contributes valuable 
information to the 
discussion with minor 
clarity or mechanics 
errors. 

Post unorganized or rude 
content that contains 
multiple errors. 
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